Exact solutions for diluted spin glasses and optimization problems 
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We study the low temperature properties of p-spin glass models with finite connectivity and 
of some optimization problems. Using a one-step functional replica symmetry breaking Ansatz we 
can solve exactly the saddle-point equations for graphs with uniform connectivity. The resulting 
ground state energy is in perfect agreement with numerical simulations. For fluctuating connectivity 
graphs, the same Ansatz can be used in a variational way: For p-spin models (known as p-XOR- 
SAT in computer science) it provides the exact configurational entropy together with the dynamical 
and static critical connectivities (for p = 3, 7^ = 0.818 and y s — 0.918 resp.), whereas for hard 
optimization problems like 3-SAT or Bicoloring it provides new upper bounds for their critical 
thresholds ( 7 ™ r = 4.396 and 7 r r = 2.149 resp.). 



PACS Numbers : 75.10.Nr, 89.80.+h 

The nature of the glassy phase and of the out-of- 
equilibrium dynamics of physical systems are two inter- 
twined aspects of the behavior of many complex systems 
found in different fields, ranging from physics or biology 
to computer science and game theory. The existence and 
the characterization of long time states, and the question 
of relaxation times are important open issues of modern 
statistical mechanics and probability theory. 

Fully connected spin glasses have served as prototype 
models able to provide a highly non trivial static and 
off-equilibrium phenomenology already at the mean-field 
level Jl],^| . However, some important features of com- 
plex real-world systems heavily rely on the connectivity 
pattern which is particularly simple in these systems. 

For instance, super-cooled liquids and structural 
glasses are characterized by a finite number of short range 
interactions for each particle (bounded by the so called 
kissing number of the particles), which leads to a complex 
structure of energy and entropy barriers in phase space. 
Heterogeneities in both the static and the off-equilibrium 
regimes witness such underlying constraints 

In computer science, non-trivial ensembles of hard 
combinatorial optimization problems, the so called NP- 
complete problems Q, typically map onto spin glasses 
with finite average connectivity (or degree) at zero tem- 
perature ||. In the last years methods from statistical 
physics have been very useful in order to study phase 
transitions in such problems The hardest among 

these share characteristics that are largely independent 
on the specific algorithms adopted for their solution. Im- 
portant features such as solution time or memory re- 
quirements are conjectured to be strictly related to the 
geometrical structure of their low temperature phase 
space (7). For a survey on the state of the art we ad- 
dress the reader to two recent special issues || . 

The main technical obstacle for the development of a 
statistical physics theory over finite degree graphs has 
been the extension of the Parisi replica symmetry break- 



ing (RSB) scheme to the functional level fifTO]. The 
replica symmetric (RS) phases are described by a single 
probability distribution function of the effective fields, 
capturing the site-to-site fluctuations of the local magne- 
tization. On the contrary, when the symmetry among 
replicas breaks down, there appear many pure states, 
each one endowed with its own effective field distribution. 
The site-to-site fluctuations induce correlations among 
such probability distributions. The overall free energy 
has to be optimized in a large functional space which 
becomes more and more complex as the Parisi scheme 
is iterated. Recently in flT|| a population dynamics algo- 
rithm has been introduced which is able, for large enough 
computer resources, to reconstruct the field distributions 
that lead to the numerical solution of the one-step RSB 
(1RSB) equations in full generality. 

In this Letter we re-examine in the appropriate 1RSB 
context an Ansatz previously introduced in jlO| to get an 
approximated solution of the Viana-Bray model. That 
Ansatz, which neglects site-to-site fluctuations, allows for 
great simplifications in the functional equations describ- 
ing the low temperature regime of spin glasses and op- 
timization problems defined on graphs with finite con- 
nectivity. The Ansatz solves exactly the saddle-point 
equations for models defined over uniform degree ran- 
dom hypergraphs and it is simple enough to allow for 
the explicit computation of the thermodynamic quanti- 
ties at low temperature, which remarkably agree with 
numerics. As representative instances, we compute the 
ground state (GS) energy of the p-spin glass and of the 
Bicoloring problem. In both cases, the problem of find- 
ing the GS is a hard computational task (NP-hard). To 
the best of our knowledge, the one presented here is the 
first exact, fully analytical solution for diluted models in 
the RSB phase. 

For the p-spin glass with fluctuating connectivities 
where site-to-site correlations are in principle important, 
the same Ansatz allows to evaluate exactly the configu- 
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rational entropy and the dynamical and static transition 
points found as the average connectivity is increased |L2| . 
In computer science this model is known as random p- 
XOR-SAT and its complete probabilistic character- 
ization is considered an open problem (at present only 
lower and upper rigorous bounds to the critical threshold 
are known [fL4|). From a combinatorial standpoint this 
problem has straight connections with the well-studied 
domain of random linear systems over finite fields, with 
applications in coding and cryptography pl| . 

In the fluctuating connectivity framework, we have also 
tested the method as a variational approach to the study 
of the GS properties of random NP-complete combinato- 
rial problems, such as random 3-SAT or Bicoloring jjj, 
obtaining the currently most accurate analytical estima- 
tion of their critical SAT/UNSAT thresholds (see below). 

In what follows we shall concentrate on the p = 3 
spin interactions, the generalization to arbitrary p be- 
ing straightforward. The Hamiltonian of the models we 
have chosen to study reads H = j k}&E G[Si, Sj, Sk], 
where Si = ± 1 are Ising spins and E is the set of triples 
(plaquettes) , which form a random hypergraph (locally 
the graph has the topology of a Husimi tree). In the 
fixed connectivity case, every index i = 1, . . . , N must 
appear in E the same number k+l of times. However the 
presence of hyperloops |l2| of length of the order \og(N) 
induce non trivial contributions to the free-energy. In the 
fluctuating connectivity case, each possible plaquette is 
chosen at random with probability j/N 2 . 

In the 3-spin glass case, the local interaction reads 
G[Si,Sj,Sk] = -JijkSiSjSk with Jij k = ±1 randomly. 
For the Bicoloring problem we have G[Si, Sj, Sk] = 
SiSuSjWSnSk) = frSiSj + S % S k + SiSj + 1). Each 
interaction adds zero energy only if the spin configura- 
tion is not monochromatic, i.e. not all spins on the pla- 
quette are equal. The optimization problem amounts at 
minimizing the number of monochromatic plaquettes. 

Firstly we focus on the fixed connectivity case (with 
connectivity k+l). In order to calculate the averaged free 
energy at inverse temperature (3 we resort to the replica 
method, where the average free energy is evaluated from 
an analytic continuation of the integer moments of the 
partition function jl|] . The replicated free energy reads 
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with Q[x,y,z] — xyz for the spin glass and Q[x, y, z] = 
— (xy + yz + xz + l)/4 for Bicoloring. a = 1, .., n is the 
replica index, a = {<x , .., a n } is a vector of n Ising vari- 
ables and /((?) (x c(a) k , where c(a) = N~ x S(Si — a) 
counts the fraction of sites i having replicated spin Sf — 



a a . The functional order parameter /(<?) must be sym- 
metric in a . From Eq. (|l|) one obtains the following saddle 
point equations 
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The above equations admit a paramagnetic and a spin 
glass solution. As long as replica symmetry holds (e.g. 
in the paramagnetic or ferromagnetic phases) the order 
parameter /(<?) depends only on the sum J2 a aa i i- e - on 
a single probability distribution of effective local fields. 
However, in the glassy phase the RS solution is not opti- 
mal (and yet stable), and one needs a RSB Ansatz, which 
is in general very complicated, even in the simplest case 
of 1RSB. Nevertheless in our case, the observation that 
the sites are locally equivalent suggests to neglect site-to- 
site fluctuations in the distribution of the effective fields. 
In the replica formalism this fact is reflected in the use 
of the following factorized Ansatz p0| , p7[ 
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m = n f(p a ) = n / dhp{h) 
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, (4) 



where the n replicas have been divided into n/m groups 
of m replicas each and a g is an m-dimensional vector 
with the components belonging to the g-th. group. The 
general interpretation of the 1RSB order parameter Jll]] 
shows that P(h) is the probability distribution of the 
cavity field, i.e. the field on a site after one of the inter- 
actions of that site has been removed. The above Ansatz 
is consistent with the saddle point equations ((2||), and 
the very same equations are verified by /((? 9 ) and D(a g ) 
with the sum in (|^) running only up to m. 

The RSB saddle point equations are different for the 
two models. In the 3-spin case we find 
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where Q(u) = J VhVgS[u-u(h,g)}, with Vh = dhP(h) 
and t&nh[(3u(h, g)] — tanh(/3) tanh(/3/i) tanh(/?g), and Ak 
normalizes the P{h). For Bicoloring we find 
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where t(h,g) = |/3 1 ln(a_/a+) and W(h,g) = ^/a_a + 
witha± = 2cosh[/3(h-.9)] + 2cosh[/3(fr + .g±I)]e~ /3 . The 
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RS (resp. paramagnetic) equation is recovered for m = n 
(resp. m = 1). As usual, in order to find the thermo- 
dynamical free-energy a maximization of the free-energy 
functional with respect to m should be performed. 

For a generic temperature the solutions to @ and (||) 
can be easily found numerically with a RS-like popula- 
tion dynamics, which requires much less computational 
effort than the 1RSB algorithm of Ref. @. 

Interestingly enough in the limit of zero tempera- 
ture we can solve the equations analytically. Indeed 
for (3 — > oo we have u(h,g) — > sign(/i g) min(l, h, g) and 
t(h,g) — > — sign(/i) min(l, h, g) 0(hg) (where 9 is the step 
function). Then both u(h,g) and t(h,g) can take only 
the values and ±1 for integer valued cavity fields. Ra- 
tional valued solutions also exist and yet vanish close to 
the free energy maximum. 

For the p-spin with odd connectivity k + 1 and for the 
Bicoloring the analytical expressions are quite involved 
and will be given elsewhere Jl8| . For p-spin with even 
connectivities (i.e. odd k) the solution can be written in 
a very compact way, 
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with g(y, k) = — In 
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where y is the zero temperature limit of the quantity (3m 
which turns out to be finite. The GS energy (e ffs ) cor- 
responds to the maximum of fo [|l|. For connectivities 
smaller than 4 for the p-spin and 7 for the Bicoloring the 
maximum is always in y = oo and corresponds to the RS 
paramagnetic solution P(h) = 5(h). The RS spin glass 
solution, located in y = 0, has always a lower energy 
with respect to the physical one. For some connectivities 
the free energy values (e gs ) are reported in the tables, to- 
gether with the corresponding y — y* saddle-point values. 
For the 3-spin case we also report numerical estimations 
of the GS energy (e™" m ) obtained by extrapolating the 
results of exhaustive enumerations (sizes up to N = 60 
averaged over 1000-10000 samples). Moreover in jl6| the 
y* value for the 3-spin model with k + 1 = 4 has been 
estimated to be 1.41(1), perfectly compatible with our 
analytic value. 

A further check of the analytic solution in the p-spin 
case is provided by the proper convergence, in the limit 
k — > oo, to the exact solution of the fully connected p- 
spin model by Gardner |l9|] after a proper rescaling of 
the coupling. 
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In the case of models defined over non-homogeneous 
graphs the Ansatz can be used to obtain approximate or, 
in some cases, exact variational estimates of the thermo- 
dynamic functions. We have considered three representa- 
tive cases: the 3-spin model and the Bicoloring problem 
over random hypergraphs with a Poisson distribution of 
site connectivities and the random 3-SAT problem. 

For the 3-spin model we obtained estimates of the dy- 
namical and static critical points looking at the configu- 
rational entropy. It has been shown in ]l2] | that in this 
model, the GS are clustered. Given a GS there is an 
exponential number of other GS which can be reached 
through GS paths where subsequent GS differ only by 
a finite number of spin flips. For small average connec- 
tivity there is a unique cluster, while above a threshold 
7d the number of disconnected cluster become exponen- 
tially large. The configurational entropy is the logarithm 
of the number of cluster per spin, and can be computed 

in the replica 1RSB formalism as £(7) = jf- [ po| . 

£(7) jumps to a non zero value at the dynamical critical 
point jd and then it vanishes again at the static critical 
point 7s. Using arguments put forward in pi}| , one can 
show [ |18| that, due to the triviality of the paramagnetic 
phase of the model where P(h) — 5(h), the factor- 
ized Ansatz (|j) yields the exact result. The resulting 
expression reads 



£(7) = ln(2)[r - 37r 2 (l - r) - 7 r 3 



(8) 



with r solving the m — I saddle point equation 1 — r = 
cxp(— 37r 2 ). The above expression is different from zero 
between 7^ = 0.818 and 7^ = 0.918 (see bold line in Fig. |l|) 
where it equals the difference between the paramagnetic 
and the ferromagnetic entropies |12|. The above critical 
points coincide with numerical estimates 

In this model the static RSB transition point 7^ co- 
incides with the critical point 7 C beyond which the sys- 
tem becomes frustrated, that is, no longer all interactions 
can be satisfied at the same time and therefore the GS 
energy becomes positive [p^| . In computer science this 
point is known as the SAT/UNSAT critical threshold: 
7 C = 7 S = 0.918 thus provides the critical threshold for 
random 3-XOR-SAT, in perfect agreement with the nu- 



merical estimation |12 



Above 7s the factorized Ansatz ceases to be exact and 
can be used only at a variational level. 
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FIG. 1. Ground states configurational entropy versus mean 
connectivity for the 3-spin. The results of numerical cluster- 
ing with an overlap cut-off of 0.7 (averaged over 1000, 1000, 
500 and 50 samples) converge to the analytical prediction. 

In order to check numerically Eq.(^), we have per- 
formed a GS clustering. This task is in general very hard 
due to the large number of GS and because of the lack 
of a proper definition of clusters in a finite size systems. 
However in this model calculations are easier, thanks to 
the presence of only two relevant overlaps between GS: 
The internal overlap equals r and is always larger than 
0.7, while different clusters are almost orthogonal. This 
leads to an optimal cut-off of 0.7 for the numerical iden- 
tification of clusters. The results shown in Fig. [l] are in 
remarkable agreement with the analytical curve. 

Finally, in models where the configurational entropy 
is likely to be zero and correlation among clusters are 
stronger, like Bicoloring over fluctuating degree random 
graphs and 3-SAT the dynamic and static critical 
points coincide (jd = j s ) and precede the SAT/UNSAT 
critical point 7 C (usually called a c ). Though less effec- 
tive, the factorized Ansatz with integer valued fields still 
provides an estimate for 7 C Namely 7™ r = 4.3962 

for 3-SAT and j^ ar = 2.149 for Bicoloring, which improve 
present rigorous bounds (recently reviewed in 24 1). 

Summarizing, we have studied a factorization Ansatz 
which allows to solve exactly diluted spin-glass and op- 
timization models, on homogeneous hypergraphs. These 
analytical results can play a crucial role in testing con- 
vergence of heuristic algorithms. For non-homogeneous 
graphs the Ansatz still allows to get very good results 
(some of them exact) as we have shown for the p-spin, 
the Bicoloring and the 3-SAT problems. Encouraged by 
the recent rigorous results obtained on simpler models 
in |^,^|, we trust that an alternative and mathemati- 
cally rigorous derivation of our results may be possible. 

We thank M. Mezard, G. Parisi and M. Weigt for sev- 
eral useful discussions. 
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